Older adults fail to form stable task representations during model-based reversal inference

Older adults struggle in dealing with changeable and uncertain environments across several cognitive domains. This has been attributed to difficulties in forming adequate task representations that help navigate uncertain environments. Here, we investigate how, in older adults, inadequate task representations impact on model-based reversal learning. We combined computational modeling and pupillometry during a novel model-based reversal learning task, which allowed us to isolate the relevance of task representations at feedback evaluation. We find that older adults overestimate the changeability of task states and consequently are less able to converge on unequivocal task representations through learning. Pupillometric measures and behavioral data show that these unreliable task representations in older adults manifest as a reduced ability to focus on feedback that is relevant for updating task representations, and as a reduced metacognitive awareness in the accuracy of their actions. Instead, the data suggested older adults' choice behavior was more consistent with a guidance by uninformative feedback properties such as outcome valence. Our study highlights that an inability to form adequate task representations may be a crucial factor underlying older adults' impaired model-based inference

Evidence for surprise minimization over value maximization in choice behavior

Classical economic models are predicated on the idea that the ultimate aim of choice is to maximize utility or reward. In contrast, an alternative perspective highlights the fact that adaptive behavior requires agents' to model their environment and minimize surprise about the states they frequent. We propose that choice behavior can be more accurately accounted for by surprise minimization compared to reward or utility maximization alone. Minimizing surprise makes a prediction at variance with expected utility models; namely, that in addition to attaining valuable states, agents attempt to maximize the entropy over outcomes and thus 'keep their options open'. We tested this prediction using a simple binary choice paradigm and show that human decision-making is better explained by surprise minimization compared to utility maximization. Furthermore, we replicated this entropy-seeking behavior in a control task with no explicit utilities. These findings highlight a limitation of purely economic motivations in explaining choice behavior and instead emphasize the importance of belief-based motivations

A computational analysis of abnormal belief-updating processes and their association with psychotic experiences and childhood trauma in a UK birth cohort

BACKGROUND: Psychotic experiences emerge from abnormalities in perception and belief formation, and occur more commonly in those experiencing childhood trauma. Yet, which precise aspects of belief formation are atypical in psychosis is not well understood. We used a computational modelling approach to characterise belief-updating in young adults in the general population, examine their relationship with psychotic outcomes and trauma, and the extent to which they mediate the trauma-psychosis relationship. METHODS: We used data from 3,360 individuals from the Avon Longitudinal Study of Parents and Children birth cohort who completed assessments for psychotic outcomes, depression, anxiety, and two belief-updating tasks at age 24, and had data available on traumatic events assessed from birth to late adolescence. Unadjusted and adjusted regression and counterfactual mediation methods were used for the analyses. RESULTS: Basic behavioural measures of belief-updating ('draws to decision' and 'disconfirmatory updating') were not associated with psychotic experiences. However, computational modelling revealed an association between increased decision noise with both psychotic experiences and trauma exposure, although <3% of the trauma-psychotic experience association was mediated by decision noise. Belief-updating measures were also associated with intelligence and socio-demographic characteristics, confounding most of the associations with psychotic experiences. There was little evidence that belief-updating parameters were differentially associated with delusions compared to hallucinations, or that they were differentially associated with psychotic outcomes compared to depression or anxiety. CONCLUSIONS: These findings challenge the hypothesis that atypical belief-updating mechanisms (as indexed by the computational models and behavioural measures we employed) underlie the development of psychotic phenomena

Beliefs and desires in the predictive brain

Bayesian brain theories suggest that perception, action and cognition arise as animals minimise the mismatch between their expectations and reality. This principle could unify cognitive science with the broader natural sciences, but leave key elements of cognition and behaviour unexplained

Lateral Orbitofrontal Cortex Involvement in Initial Negative Aesthetic Impression Formation

It is well established that aesthetic appreciation is related with activity in several different brain regions. The identification of the neural correlates of beauty or liking ratings has been the focus of most prior studies. Not much attention has been directed towards the fact that humans are surrounded by objects that lead them to experience aesthetic indifference or leave them with a negative aesthetic impression. Here we explore the neural substrate of such experiences. Given the neuroimaging techniques that have been used, little is known about the temporal features of such brain activity. By means of magnetoencephalography we registered the moment at which brain activity differed while participants viewed images they considered to be beautiful or not. Results show that the first differential activity appears between 300 and 400 ms after stimulus onset. During this period activity in right lateral orbitofrontal cortex (lOFC) was greater while participants rated visual stimuli as not beautiful than when they rated them as beautiful. We argue that this activity is associated with an initial negative aesthetic impression formation, driven by the relative hedonic value of stimuli regarded as not beautiful. Additionally, our results contribute to the understanding of the nature of the functional roles of the lOFC

Neural correlates of evidence accumulation during value-based decisions revealed via simultaneous EEG-fMRI

Current computational accounts posit that, in simple binary choices, humans accumulate evidence in favour of the different alternatives before committing to a decision. Neural correlates of this accumulating activity have been found during perceptual decisions in parietal and prefrontal cortex; however the source of such activity in value-based choices remains unknown. Here we use simultaneous EEG–fMRI and computational modelling to identify EEG signals reflecting an accumulation process and demonstrate that the within- and across-trial variability in these signals explains fMRI responses in posterior-medial frontal cortex. Consistent with its role in integrating the evidence prior to reaching a decision, this region also exhibits task-dependent coupling with the ventromedial prefrontal cortex and the striatum, brain areas known to encode the subjective value of the decision alternatives. These results further endorse the proposition of an evidence accumulation process during value-based decisions in humans and implicate the posterior-medial frontal cortex in this process

Polarity of uncertainty representation during exploration and exploitation in ventromedial prefrontal cortex

Environments furnish multiple information sources for making predictions about future events. Here we use behavioural modelling and functional magnetic resonance imaging to describe how humans select predictors that might be most relevant. First, during early encounters with potential predictors, participants’ selections were explorative and directed towards subjectively uncertain predictors (positive uncertainty effect). This was particularly the case when many future opportunities remained to exploit knowledge gained. Then, preferences for accurate predictors increased over time, while uncertain predictors were avoided (negative uncertainty effect). The behavioural transition from positive to negative uncertainty-driven selections was accompanied by changes in the representations of belief uncertainty in ventromedial prefrontal cortex (vmPFC). The polarity of uncertainty representations (positive or negative encoding of uncertainty) changed between exploration and exploitation periods. Moreover, the two periods were separated by a third transitional period in which beliefs about predictors’ accuracy predominated. The vmPFC signals a multiplicity of decision variables, the strength and polarity of which vary with behavioural context

Biased belief updating and suboptimal choice in foraging decisions

Deciding which options to engage, and which to forego, requires developing accurate beliefs about the overall distribution of prospects. Here we adapt a classic prey selection task from foraging theory to examine how individuals keep track of an environment’s reward rate and adjust choices in response to its fluctuations. Preference shifts were most pronounced when the environment improved compared to when it deteriorated. This is best explained by a trial-by-trial learning model in which participants estimate the reward rate with upward vs. downward changes controlled by separate learning rates. A failure to adjust expectations sufficiently when an environment becomes worse leads to suboptimal choices: options that are valuable given the environmental conditions are rejected in the false expectation that better options will materialize. These findings offer a previously unappreciated parallel in the serial choice setting of observations of asymmetric updating and resulting biased (often overoptimistic) estimates in other domains

The role of human orbitofrontal cortex in value comparison for incommensurable objects.

The human orbitofrontal cortex is strongly implicated in appetitive valuation. Whether its role extends to support comparative valuation necessary to explain probabilistic choice patterns for incommensurable goods is unknown. Using a binary choice paradigm, we derived the subjective values of different bundles of goods, under conditions of both gain and loss. We demonstrate that orbitofrontal activation reflects the difference in subjective value between available options, an effect evident across valuation for both gains and losses. In contrast, activation in dorsal striatum and supplementary motor areas reflects subjects' choice probabilities. These findings indicate that orbitofrontal cortex plays a pivotal role in valuation for incommensurable goods, a critical component process in human decision making